Fault-tolerant computer system, fault-tolerant computer system control method and recording medium storing control program for fault-tolerant computer system

ABSTRACT

In a fault-tolerant computer system that includes a computer  1 A including a virtual machine on which a guest OS  3 A is operating and a computer  1 B in operation; the computer  1 A comprises a snapshot manager  8 A that saves a guest OS snapshot  10 A, which is a difference information at each checkpoint, of the guest OS  3 A in a memory  4 A of the computer  1 A, and sends an instruction at each checkpoint to copy the guest OS snapshot  10 A as a guest OS snapshot  10 B to a memory  4 B of a computer  1 B via transfer unit  11 A and  11 B; and that computer  1 B comprises a snapshot manager  8 B that activates a guest OS  3 B based on the guest OS snapshot  10 B when the computer  1 A stops.

This application is based on Japanese Patent Application No. 2011-087745filed on Apr. 11, 2011 and including specification, claims, drawings andsummary. The disclosure of the above Japanese Patent Application isincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to a fault-tolerant computer system, afault-tolerant computer system control method and a recording mediumstoring control program for fault-tolerant computer system.

BACKGROUND ART

In recent years, virtualization technology that makes it possible tooperate a plurality of Operating Systems (OS) on a physical machine hasbeen widely used. For achieving a virtual machine, there is a methodwherein a layer is created in the OS (host OS) that operates on atypical physical machine for operating a virtual machine, and there is amethod of creating a layer (hypervisor) on the hardware for operating avirtual machine without going through the host OS, and operating a guestOS on that layer.

Japanese Patent No. 4,468,426 discloses a method of collectingsynchronization information that was generated for a first virtualmachine and that relates to an event that accompanies input to the firstvirtual machine, and controlling the execution state of the input of asecond virtual machine according to that synchronization information sothat it is the same as the execution state of the input of the firstvirtual machine.

Japanese Patent Application No. 2009-080692 discloses a method wherein,when failure occurs in a server computer on which a virtual machine isoperating, the virtual machine is reproduced on another server computerbased on a snapshot that was obtained by a disk drive at the closestpoint in time to the time when the failure occurred. This snapshot isCPU context of the computer that is operating and data inside the memoryfor processing that is used when that CPU is operating, and data insidethe disk drive that is extracted at predetermined timing (check point).

Japanese Patent Application No. 2008-033483 discloses a method wherein,when failure occurs, the list of files included in a copy image on thedisk of a first computer and the execution context of the computer arecopied to a second computer, that list of files is then referenced inorder to copy the copy image from the disk of the first computer to thedisk of the second computer.

SUMMARY

An exemplary object of the present invention is to provide a low-powerconsuming fault-tolerant computer system, a fault-tolerant computersystem control method and recording medium storing control program forfault-tolerant computer system capable of quick and simple systemswitching.

In order to accomplish the exemplary object above, the fault-tolerantcomputer system of a first exemplary aspect of the present invention is

a fault-tolerant computer system that includes a first computer thatincludes a first memory and a first transfer unit, and operates avirtual machine including a guest OS; and a second computer thatincludes a second memory and a second transfer unit that receives datathat is transferred from the first transfer unit; wherein

the first computer includes

a first snapshot manager that, together with acquiring a snapshot of thevirtual machine at each predetermined first timing, causes to save adifference information in the first memory of a snapshot of the virtualmachine at the first timing from one previous first timing, and sends aninstruction to the first transfer unit to transfer the differenceinformation that is saved in the first memory to the second memory viathe first transfer unit and the second transfer unit; and

the second computer includes

a second snapshot manager that, together with generating the snapshotbased on the difference information that was transferred to the secondmemory via the first transfer unit and the second transfer unit andsaving that snapshot in the second memory, activates a guest OS by thesecond computer at a predetermined second timing based on the snapshotthat was saved in the second memory.

The control method for a fault-tolerant computer system of a secondexemplary aspect of the present invention is

a control method for a fault-tolerant computer system that includes afirst computer that includes a first memory and a first transfer unit,and operates a virtual machine comprising a guest OS, and a secondcomputer that includes a second memory and a second transfer unit thatreceives data that is transferred from the first transfer unit;

acquiring a snapshot of the virtual machine at each predetermined firsttiming, and saves a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing,

transferring the difference information that is saved in the firstmemory to the second memory via the first transfer unit and the secondtransfer unit;

generating a snapshot based on the difference information that wastransferred to the second memory, and saves that snapshot in the secondmemory; and

activating a guest OS by the second computer at a predetermined secondtiming based on the snapshot that was saved in the second memory.

The recording medium storing control program for fault-tolerant computersystem of a third exemplary aspect of the present invention is

a non-transitory recording medium that stores a control program for afault-tolerant computer system that includes a first computer thatincludes a first memory and a first transfer unit, and operates avirtual machine comprising a guest OS, and a second computer, and causes

the first computer to

acquire a snapshot of the virtual machine at each predetermined firsttiming, and save a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing, and

transfer the difference information that is saved in the first memory tothe second computer via the first transfer unit.

The recording medium storing control program for fault-tolerant computersystem of a fourth exemplary aspect of the present invention is

a non-transitory recording medium that stores a control program for afault-tolerant computer system that includes a first computer thatoperates a virtual machine comprising a guest OS, and a second computerthat includes a second transfer unit that receives data that istransferred from the first computer and a second memory that saves thetransferred data; wherein

the transferred data are a difference information of snapshots of thevirtual computer; the non-transitory recording medium that stores acontrol program for a fault-tolerant computer system that causes

the second computer to

generate a snapshot based on the difference information that was savedin the second memory, and save that snapshot in the second memory, and

activate a guest OS at a predetermined second timing based on thesnapshot that was saved in the second memory.

The fault-tolerant computer system of a fifth exemplary aspect of thepresent invention is

a fault-tolerant computer system that includes a first computer thatincludes a first memory and first transfer means, and operates a virtualmachine comprising a guest OS; and a second computer that includes asecond memory and second transfer means that receives data that istransferred from the first transfer means; wherein

the first computer includes

a first snapshot managing means that, together with acquiring a snapshotof the virtual machine at each predetermined first timing, causes tosave a difference information in the first memory of a snapshot of thevirtual machine at the first timing from one previous first timing, andsends an instruction to the first transfer means to transfer thedifference information that is saved in the first memory to the secondmemory via the first transfer means and second transfer means; and

the second computer includes

a second snapshot managing means that, together with generating thesnapshot based on the difference information that was transferred to thesecond memory via the first transfer means and the second transfer meansand saving that snapshot in the second memory, activates a guest OS bythe second computer at a predetermined second timing based on thesnapshot that was saved in the second memory.

BRIEF DESCRIPTION OF THE DRAWINGS

These objects and other objects and advantages of the present inventionwill become more apparent upon reading of the following detaileddescription and the accompanying drawings in which:

FIG. 1 is a block diagram illustrating an example of the minimumconstruction of a fault tolerant computer system of an exemplaryembodiment of the present invention;

FIG. 2 is a block diagram illustrating an example of the construction ofa fault-tolerant computer system of an exemplary embodiment;

FIG. 3A is a flowchart illustrating process A of an active system of thecomputer switching process of a fault-tolerant computer system of anexemplary embodiment;

FIG. 3B is a flowchart illustrating process B of an active system of thecomputer switching process of a fault-tolerant computer system of anexemplary embodiment;

FIG. 4A is a flowchart illustrating process C of a standby system of thecomputer switching process of a fault-tolerant computer system of anexemplary embodiment;

FIG. 4B is a flowchart illustrating process D of a standby system of thecomputer switching process of a fault-tolerant computer system of anexemplary embodiment;

FIG. 5 is a diagram illustrating an example of setting checkpoints ofthe computer switching process of a fault-tolerant computer system of anexemplary embodiment;

FIG. 6 is a diagram illustrating another example of setting checkpointsof the computer switching process of a fault-tolerant computer system ofan exemplary embodiment;

FIG. 7 is a diagram illustrating another example of setting checkpointsof the computer switching process of a fault-tolerant computer system ofan exemplary embodiment; and

FIG. 8 is a block diagram illustrating an example of a variation ofconstruction of a fault-tolerant computer system of an exemplaryembodiment.

EXEMPLARY EMBODIMENTS

The fault-tolerant computer system of this embodiment of the presentinvention includes at least two physical machines and is comprised suchthat virtual machines operate on each physical machine. In thefollowing, an example in which the system has two physical machines willbe explained. Here, the term physical machine is for distinguishing themachine from a virtual machine, and means an actual computer. FIG. 1 andFIG. 2 illustrate examples of the construction of a fault-tolerantcomputer system. As illustrated in FIG. 1 and FIG. 2, one of the twophysical machines of the fault-tolerant computer system is called activesystem 1A, and the other is called standby system 1B. The active system1A is a computer that has a virtual machine that is operating andproviding a service to a user. The standby system 1B is a computer thathas a virtual machine that is waiting to start operation when failureoccurs and the system is switched (failed over). The component elementsof the virtual machines in both systems are basically the same.

First, FIG. 1 will be explained. FIG. 1 illustrates an example of theminimum construction of a fault-tolerant computer system of an exemplaryembodiment of the present invention.

The active system 1A comprises a host OS (omitted in the figure), aguest OS 3A, a memory 4A, a snapshot manager 8A and a transfer unit 11A,and by the operation of the guest OS 3A, functions as a virtual machine.

The standby system 1B comprises a host OS (omitted in the figure), amemory 4B, a snapshot manager 8B and a transfer unit 11B, and operatesas an actual computer. The guest OS 3B does not operate at first, so isillustrated by a dashed line.

The memory 4A can be accessed from both the host OS and the guest OS 3A.The memory 4B can be accessed from the host OS of the standby system 1B.After the guest OS 3B has been activated in the standby system 1B, thememory 4B can also be accessed from the guest OS 3B.

The snapshot manager 8A is operated on the host OS and guest OS 3A, andthe difference information of a snapshot of the guest OS 3A, which willbe described later, saves at predetermined timing in the memory 4A as aguest OS snapshot 10A. A snapshot is a matter that stores structures ofstorage (memory and the like) and files and the execution state ofprocessing at a certain point.

The transfer unit 11A transfers the guest OS snapshot 10A that is storedin the memory 4A to the standby system 1B.

The transfer unit 11B receives the guest OS snapshot 10A that wastransferred from the transfer unit 11A, and saves that guest OS snapshot10A in memory 4B as guest OS snapshot 10B.

The snapshot manager 8B is operated on the Host OS, and based on theguest OS snapshot 10B that is stored in memory 4B, generates a completeguest OS snapshot 10B that is integrated with the differenceinformation, and saves that complete guest OS snapshot 10B in memory 4B.

Moreover, the snapshot manager 8B, when it was determined atpredetermined timing, which will be described later, that the activesystem 1A stopped, for example, activates a predetermined program, andactivates the guest OS 3B based on this complete guest OS snapshot 10B.Due to the activation of the guest OS 3B, the standby system 1B beginsto operate as the active system 1A in which the virtual machinefunctions.

FIG. 2 illustrates active system 1A and standby system 1B in more detailthan in FIG. 1. In FIG. 2, the host OS 2A, 2B are illustrated. Thisexemplary embodiment will be explained in detail according to FIG. 2.

The active system 1A comprises a host OS 2A that manages the operationof the active system 1A, a guest OS 3A, which is the OS of the virtualmachine, a memory 4A that can be accessed from both the host OS 2A andthe guest OS 3A, a FT (Fault Tolerant) module 5A, a RDMA (Remote DirectMemory Access) driver 6A and a data transfer unit 7A. The CPU (CentralProcessing Unit, omitted in the figure) of the hardware of the activesystem 1A functions as the guest OS 3A and host OS 2A by using RAM(Random Access Memory), ROM (Read Only Memory) and/or the like andexecuting predetermined programs for each. The same is true for the FTmodule 5A and RDMA (Remote Direct Memory Access) driver 6A. The RDMAdriver 6A and data transfer unit 7A form the transfer unit 11A. The hostOS 2A and the guest OS 3A are connected via a hypervisor, for example.Depending on the method used for the virtual machine, instead of beingconnected by a hypervisor, the guest OS 3A can be operated on the hostOS 2A.

The standby system 1B comprises a host OS 2B that manages the operationof the standby system 1B, a memory 4B that can be accessed by both thehost OS 2B and activated guest OS 3B, a FT module 5B, a RDMA driver 6Band a data transfer unit 7B. The CPU of the hardware of the standbysystem 1B functions as the guest OS 3B and host OS 2B by using RAM. ROM(both omitted in the figure) and/or the like and executing apredetermined program for each. The same is true for the FT module 5Band the RDMA driver 6B. The guest OS 3B is not activated yet. Therefore,the guest OS 3B is indicated with a dashed line in FIG. 2. The RDMAdriver 6B and the data transfer unit 7B form the transfer unit 11B.

Both the transfer unit 11A and the transfer unit 11B comprise acommunication unit, and the active system 1A and standby system 1B areconnected together via those communication units. More specifically, thedata transfer units 7A and 7B are connected by a communication line, anddata and various kinds of information can be exchanged over thatcommunication line.

The memories 4A, 4B are memory devices of the physical machines, and aremain memory devices, for example, that are capable of writing or readinginformation at higher speed than an external storage such as a diskdrive that is often used currently. The memories 4A, 4B store programsthat are being executed, data and guest OS snapshots, which will bedescribed later.

The FT module 5A is a module that has a function for making possible afault-tolerant computer system. The FT module 5A comprises a snapshotmanager 8A and error handler 9A, and is operated on the host OS 2A. Inother words, the CPU of the active system 1A functions as the snapshotmanager 8A and error handler 9A on host OS 2A by executing programsrelated to each.

The snapshot manager 8A manages check points, acquires and manageschange (rewritten) information for the saved contents in the memory thatthe guest OS 3A uses and manages, and sets the interval for transferringguest OS snapshots 10A (described later). Moreover, the snapshot manager8A, in accordance to the set transfer interval, sets the contents of theguest OS snapshot 10A that will be saved in memory 4A and saves theresults in memory 4A, and gives an instruction to transfer the guest OSsnapshot 10A that is saved in the memory 4A to the RDMA driver 6A.

The method for acquiring change information for the contents saved inthe memory that guest OS 3A uses and manages from the dirty page flag ofthe memory 4A that the CPU manages is a typically know acquisitionmethod, so an explanation about the details of the specific acquisitionmethod is omitted.

The guest OS snapshot 10A is a snapshot of the computer that is operatedby the guest OS. Also, a dirty page flag is a flag that indicates thatdata is changed but not saved.

The error handler 9A, in order to notify the standby system 1B asquickly as possible that there is failure with the active system 1A,sends error information for switching systems to the standby system 1Bvia the RDMA driver 6A and data transfer unit 7A. In order to detectfailure in the active system 1A, the standby system 1B may use aheartbeat signal, for example. In that case, the error handler 9A sendsa heartbeat signal at a constant period to the standby system 1B via thedata transfer unit 7A. In the case that the heartbeat signal did notcome at a fixed interval, the data transfer unit 7B of the standbysystem 1B determines that failure occurred, or in other words,determines that operation of the active system 1A has stopped.

The RDMA driver 6A is operated on the host OS 2A. The RDMA driver 6Areceives an instruction from the FT module 5A and controls the datatransfer unit 7A so that the data transfer unit 7A transfers errorinformation or a guest OS snapshot 10A that is saved in the memory 4A tothe standby system 1B. The instruction from the FT module 5A is atransfer instruction from the snapshot manager 8A to transfer a guest OSsnapshot 10A, or an instruction from the error handler 9A to transfererror information.

The data transfer unit 7A is formed using hardware, and according tocontrol from the RDMA driver 6A, transfers a guest OS snapshot WA orerror information to the standby system 1B. More specifically, the datatransfer unit 7A receives addresses and lengths, which are necessaryinformation for copying contents stored in memory, and error informationfrom the RDMA driver 6A and transfers data according to that data.Transferring a guest OS snapshot 10A is executed by a background processof the guest OS 3A.

The data transfer unit 7B saves the guest OS snapshot 10A that wastransferred from the data transfer unit 7A in the memory 4B as a guestsnapshot 10B, and notifies the RDMA driver 6B that the transfer isfinished. The data transfer unit 7B also sends error information thatwas similarly transferred from the data transfer unit 7A, or errorinformation that was detected by the data transfer unit 7B itself to theRDMA driver 6B.

The data transfer rate between the data transfer unit 7A and transferunit 7B affects the settable checkpoint interval. A checkpoint is timingfor saving a guest OS snapshot 10A in the memory 4A, and here, is thetiming at which a guest OS snapshot 10A is transferred to the standbysystem 1B.

On the other hand, the rollback time, which is time that indicates howfar back in time the processing that is in progress during switching ofcomputers is to be returned, is affected by the checkpoint interval.

In order to reduce the rollback time as much as possible, it isnecessary to make the checkpoint interval small. Therefore, whenconnecting the data transfer units 7A, 7B, using high-speed hardwarewith a fast data transfer rate is preferred. This connection is possiblevia a typical network (with the present technology, a gigabit or 10gigabits network); however, using special hardware for performinghigh-speed DMA (Direct Memory Access) via an IO slot, such as PCIExpress is also possible. Alternatively, a method is also possible inwhich CPUs are directly connected without going through an I/O(Input/Output) slot.

The RDMA driver 6B sends the notification from the data transfer unit 7Bto the FT module 5B.

The FT module 5B, as in the active system 1A, is a module with afunction for making possible a fault-tolerant computer system. The FTmodule 5B comprises a snapshot manager 8B and error handler 9B, and isoperated on the host OS 2B.

The snapshot manager 8B and error handler 9B receive a transferredfinished notification that the guest OS snapshot 10A was saved in thememory 4B as a guest OS snapshot 10B, or receive an error information,and perform the following processing.

The error handler 9B receives the error information, determines whetherfailure occurred, or in other words, determines whether or not operationof the active system 1A has stopped, and when the judgment result isthat the operation has “stopped”, outputs a system switching signal tothe snapshot manager 8B and activates the guest OS 3B. For example, whena heartbeat signal is used, the error handler 9B, by way of the RDMAdriver 6B, determines that failure has occurred in the active system 1Awhen a heartbeat signal did not come from the active system 1A at afixed time or more, and executes error processing.

The snapshot manager 8B receives the finished notification that a guestOS snapshot 10A was saved in the memory 4B as a guest OS snapshot 10B,and by combining that guest OS snapshot 10B with the guest OS snapshots10B that have been saved in the memory 4B up to that time, saves theresult as an updated complete guest OS snapshot 10B in the memory 4B.Moreover, the snapshot manager 8B receives a switching signal from theerror handler 9B, and by activating the program for activating the guestOS 3B, activates the guest OS 3B based on the guest OS snapshot 10B thatis saved in the memory 4B. After the guest OS 3B has been activated, thestandby system takes over for the active system and executes thecontents that the snapshot manager 8A was executing in the active system1A. The host OS 2B and the guest OS 3B, as in the active system 1A, canbe connected via a hypervisor, or the guest OS 3B can be operated on thehost OS 2B.

Next, the computer switching operation of this system will be explainedusing the flowcharts illustrated in FIG. 3A, FIG. 3B, FIG. 4A and FIG.4B. FIG. 3A illustrates the contents of process A by the active system1A. FIG. 3B illustrates the contents of process B by the active system1A. FIG. 4A illustrates the contents of process C by the standby system1B. FIG. 4B illustrates the contents of process D by the standby system1B. Process A and process C are processes in the respective computerswhen copying a guest OS snapshot 10A from the active system 1A to thestandby system 1B. Process B and process D are for the acquisition andtransferring of error information by the active system 1A and processingerror information by the standby system 1B.

As a precondition, the fault-tolerant computer system, or in otherwords, active system 1A and standby system 1B are activated. Activationof the active system 1A includes both the host OS 2A and the guest OS 3Abeing in the operating state. More specifically, in the active system1A, by turning ON the power, first, the host OS 2A is set in theoperating state. The guest OS 3A is then set in the operating stateafter the host OS 2A is in the operating state. The guest OS 3A can alsobe set in the operating state by an instruction from the user. On theother hand, in the standby system 1B, by turning ON the power, the hostOS 2B is set in the operating state, however, the guest OS 3B is notactivated. The active system 1A and the standby system 1B are connectedby a communication line.

First, process A by the active system 1A will be explained based on FIG.3A. After the data transfer unit 7A acknowledges that the hardware ofboth systems are connected by a communication line, the snapshot manager8A of the active system 1A creates a snapshot of the overall processingcontents of the guest OS 3A and saves that snapshot in the memory 4A asa guest OS snapshot 10A. Moreover, the snapshot manager 8A transfersthat guest OS snapshot 10A to the standby system 1B via the transferunit 11A, or in other words, via the RDMA driver 6A and data transferunit 7A (step S10). A snapshot of the overall processing contents of theguest OS 3A includes all of the files that are used by the guest OS 3A,the contents of the processing memory at a predetermined point in time,and the context of the CPU. The method for copying the overallprocessing contents of the guest OS 3A to another system is used in themigration processing of the virtual environment and is well known, so anexplanation of that method is omitted.

Next, at a predetermined time, the snapshot manager 8A of the activesystem 1A acquires and accumulates change information related to thecontents of the guest OS snapshot 10A (step S11). The point in time whenthis predetermined time elapses is called a checkpoint (this will bedescribed in detail later). The starting point of the first checkpointis the point in time when the overall snapshot of the processingcontents of the guest OS 3A was created. In other words, at eachcheckpoint, the snapshot manager 8A acquires and accumulates the portionof change in the guest OS snapshot 10A from the starting point orprevious checkpoint up to that checkpoint.

Next, in the active system 1A, the snapshot manager 8A, for example,performs checkpoint determination (step S12). Checkpoint determinationis a determination for determining whether or not the amount of timethat has elapsed from the starting point of a checkpoint or from theprevious checkpoint has reached a predetermined time, or in other wordswhether or not the next checkpoint has been reached. The setting ofcheckpoints will be described in detail later.

When a checkpoint has not been reached (step S12: NO), processingreturns to the processing of step S11. When a checkpoint has beenreached (step S12: YES), the snapshot manager 8A saves the accumulatedchange information to which processing memory contents and CPU contextinformation at that checkpoint have been added in the memory 4A asdifference information, and outputs an instruction to the RDMA driver 6Ato transfer this difference information to the standby system 1B. Afterreceiving this instruction, the RDMA driver 6A performs control totransfer the difference information saved in the memory 4A to thestandby system 1B via the data transfer unit 7A (step S13).

After that, whether or not control was performed to stop operation isdetermined (step S14). When there was control to stop operation (stepS14: YES), processing ends. When there was no control to stop operation(step S14: NO), processing returns to step S11, and processingcontinues. Stopping operation referred to here is stopping operation bycontrol from the user, and is not the stopping of operation due to somekind of failure.

Next, the processing B in the active system will be explained based onFIG. 3B. In the active system 1A, the error handler 9A acquires error,information, and transfers the error information to the standby system1B via the data transfer unit 7A (step S20).

After that, it is determined whether control was performed to stopoperation (step S21). When control has been performed to stop operation(step S21: YES), processing ends. When there was no control to stopoperation (step S1: NO), processing returns to step S20, and processingcontinues. Stopping operation referred to here is stopping operation bycontrol from the user, and is not the stopping of operation due to somekind of failure. This process B is performed in parallel with theprocess A described above.

Next, the contents of process C in the standby system 1B will beexplained based on FIG. 4A. In the standby system 1B, the overallinformation of the guest OS snapshot 10A that was transferred from theactive system 1A in step S11 of FIG. 3A is received by the transfer unit11B, or in other words, is received by the data transfer unit 7B (stepS30). Then, according to control from the RDMA 6B, that information issaved in the memory 4B as a guest OS snapshot 10B (step S31). By doingso, the overall information of the guest OS snapshot 10A is completelycopied from the active system 1A to the standby system 1B.

After that, in the standby system 1B, the difference information thatwas transferred from the active system 1A in step S13 in FIG. 3A isreceived by the transfer unit 11B, or in other words, by the datatransfer unit 7B (step S32). Then, according to control from the RDMA6B, that difference information is saved in the memory 4B. The datatransfer unit 7B sends a notification to the snapshot manager 8B via theRDMA driver 6B indicating that saving the difference information in thememory 4B is finished. After receiving this, the snapshot manager 8Bsums up the difference information that was saved this time to the guestOS snapshot 10B that was saved up to this point, or writes over thatguest OS snapshot 10B to generate one complete guest OS snapshot 10B,and saves that complete guest OS snapshot 10B in the memory 4B (stepS33).

After that, in the standby system 1B, it is determined whether or notcontrol was performed to stop operation of the system (step S34). Whenthere was control to stop the operation (step S34: YES), the standbysystem ends processing. When there was no control to stop operation(step S34: NO), processing returns to step S32, and processingcontinues. As in the explanation of system 1A, stopping operationreferred to here is stopping operation according to control from theuser, and is not the stopping of operation due to some kind of failure.

Next, process D in the standby system 1B will be explained based on FIG.4B. In the standby system 1B, the data transfer unit 7B receives errorinformation that is transferred from the active system 1A in step S20 inFIG. 3B (step S40). The error handler 9B, based on the errorinformation, detects the occurrence of failure in the active system 1A,and determines whether or not it is necessary to switch computers (stepS41).

When heartbeat signals are used as error information, the data transferunit 7A sends heartbeat signals to the standby system 1B (step S20 inFIG. 3B). In this case, step S41 is a process wherein the error handler9B detects whether a heartbeat signal arrived in a predetermined time ormore. For the detection result, the error handler 9B determines whetheror not failure occurred in the active system 1A, and determines whetheror not it is necessary to switch computers (step S41).

When it was determined that it is not necessary to switch computers(step S41: NO), in the standby system 1B, it is determined whether ornot there was a control to stop operation of the system (step S42). Whenthere was control to stop operation (step S42: YES), the standby system1B ends the process illustrated in FIG. 4B, and when there was nocontrol to stop processing (step S42: NO), processing returns to stepS40, and processing continues.

In the standby system 1B, when the error handler 9B determined that itis necessary to switch computers (step S41: YES), the error handler 9Binstructs the snapshot manager 8B to switch computers. According to thisinstruction, the snapshot manager 8B activates the guest OS 3B based onthe guest OS snapshot 10B that is saved in the memory 4B (step S43), andends the processing illustrated in FIG. 4B. This guest OS 3B is the sameas the guest OS 3A of the point in time when previously copied from theactive system 1A. When copying of the guest OS snapshot 10A is inprogress, the case wherein the restoration of the guest OS 3B is notsufficient is possible. Therefore, preferably a plurality of copies ofthe guest OS 3A that the snapshot manager 8A manages is created, and oneguest OS snapshot 10B is maintained in the complete state, so that theguest OS 3B is always activated using a complete and new guest OSsnapshot 10B. By activating the guest OS 3B, the standby system 1Bstarts operation as a virtual machine. The guest OS 3B then continuesexecuting the processing contents at the point in time when the guest OSsnapshot 10A corresponding to the guest OS snapshot 10B that was used toactivate the guest OS 3B was created. When doing this, the screen andkeyboard connection of the physical machine are also suitably performed.Process D in FIG. 4B is performed in parallel with the process C in FIG.4A.

After operation of the active system 1A stops due to failure, and thestandby system 1B begins to function as the active system, the standbysystem executes the same processing as the active system 1A, includingthe process contents illustrated in FIG. 3A and FIG. 3B. On the otherhand, after operation of the active system was stopped due to failure,and then was reactivated after the failure was recovered, the activesystem 1A becomes the standby system and executes the processillustrated in FIG. 4A and FIG. 4B.

Next, checkpoints will be explained. Setting checkpoints is greatlyrelated to the rollback time when continuing processing by a computer.When the rollback time is long, during the time corresponding to therollback when continuing processing in the standby system, the sameprocessing is repeated two times. For example, failure occurs when auser is watching a video, the video appears to rewind and played again.Therefore, it is important to make the rollback time as short aspossible. The rollback is set with the checkpoint as an object.

In this embodiment, at first, transferring the overall snapshot of theprocessing contents of the guest OS 3A to the standby system 1B takestime. However, after that, the difference information is transferred tothe standby system 1B at each checkpoint, so the transfer time isshorter. Therefore, it is possible to make checkpoint interval smaller,and the rollback time can be made shorter by that amount. Moreover, thedifference information is not transferred all at once when failureoccurs, but is transferred at each predetermined checkpoint, so thatshortening the time required for switching computers is promoted.

Setting checkpoints will be explained for three cases.

In case 1, for example, a checkpoint is set as the point in time whenthe snapshot of the overall processing contents of the guest OS iscreated for the first time, and after that checkpoints are set at everyset amount of time. FIG. 5 is a diagram for explaining case 1. Thehorizontal axis in the figure is the time axis. In the figure, T_(i−1),T_(j), T_(i+1) are checkpoints. The interval between checkpoints is afixed time t_(c). A difference amount Q_(i) (change information) that issaved in the memory 4A as the amount of change in the contents fromcheckpoint T_(i−1) to checkpoint T_(i), or in other words, differenceinformation that includes a guest OS snapshot 10A corresponding to thedifference in memory when there was a dirty page, is transferred to thestandby system 1B and copied into the memory 4B. This copy time is setnearly by the transfer time, and depends on the amount of differencethat is the transfer object. The difference amount is the amount storedat each respective time t_(c), so this amount changes at eachcheckpoint. Therefore, the interval t_(c) between checkpoints is setlonger than the estimated maximum transfer time.

In case 1, only the difference information is transferred, so whencompared with the case of transferring the overall processing contentsof the guest OS each time, the amount of transferred information issmaller, and thus the transfer time becomes shorter. Therefore, it ispossible to make the checkpoint interval smaller, and thus it ispossible to shorten the rollback time. However, there are problems suchas the following.

For example, in FIG. 5, it is detected that failure occurred in theactive system 1A at the point in time indicated by the arrow as “failureoccurrence”, and at that point, the process for switching computers, orin other words, the system recovery process is started. At this time,the most recent guest OS snapshot 10B that is saved in the memory 4B isthe snapshot that was acquired at checkpoint T_(i), in other words, isthe portion of the amount of difference Q_(i), so that the rollback thatis necessary for the recovery process is the point in time T_(i) that isindicated as “rollback” by the arrow in FIG. 5. When the amount ofdifference Q_(i) at this time is small, the amount of time t_(i)required for copying that difference may be much smaller than the timet_(c) between checkpoints. In that case, there is a notable amount ofvacant time t_(c)−t_(i), and becomes an unnecessary rollback time.

In order to solve such a problem, checkpoints can be set based on a setamount of difference being accumulated. This is case 2. FIG. 6illustrates an example of case 2. In Case 2, the amount of difference isfixed, so the copy time is a fixed amount of time and does not depend onthe checkpoints, however, the time required to accumulate thepredetermined amount of difference differs according to the period, sothe checkpoint interval differs in length.

In this case, the checkpoints are set according to the amount ofdifference, so that the problem of vacant time that occurred in case 1does not occur. However, when the amount of difference before failureoccurs is small, and it takes a long time (t_(Ci+2)) to accumulate thepredetermined amount, the rollback time goes back one previouscheckpoint T_(i+2) before the checkpoint T_(i+3) nearest to when failureoccurred, so there are times when the rollback time may become greaterthan when the checkpoints are set at each set amount of time.

FIG. 7 illustrates case 3 of a method for setting checkpoints in orderto solve this problem. Basically, the way of thinking is the same as incase 2, however, a maximum value t_(m) is set for the time betweencheckpoints, and when the time exceeds this time, a checkpoint is set atthe point t_(m) from the previous checkpoint. The top of FIG. 7corresponds to case 2 in FIG. 6, and below that is for case 3 when amaximum time t_(m) is set between checkpoints. The rollback time hereonly goes back a time t_(m) from the nearest checkpoint T_(i+3)′ to whenfailure occurred, and when compared with case 2, the time T_(ci+2)−t_(m)becomes the reduction in the rollback.

An example was given for the case of using a heartbeat signal as themethod of detection of failure by the error handlers 7A, 7B. However, inthis method alone, when time service is stopped is the worst case, thetime of the heartbeat signal interval is added to the rollback time ofthe checkpoint. As in this exemplary embodiment, when failure occurs andit is necessary to continue the processing of the active system 1A bythe standby system 1B in a short time, it is also possible to detecterrors that could be related to future computer stoppage, and to notifythe standby system 1B using error information that includes thatinformation. As a result, the error handler 9B of the standby system 1Bcan determine that in the active system 1A the system will go down inthe near future and can startup the guest OS 3B. By doing so, thefault-tolerant computer system is able to switch the systems before thesystem stops.

In order to perform this kind of failure detection, an example ofcollecting trends of collectable errors of the memory 4A is feasible.When error information that corresponds to a collectable error trend isdetected, there is a probability that there is some kind of memoryfailure, so it is possible to determine that there is a possibility thatthe system will go down in the near future. It is possible for the errorhandlers 9A, 9B to handle various kinds of errors.

In the explanation above, it is not absolutely necessary that the errorhandlers 9A, 9B be included in the FT modules 5A, 5B. For example, asillustrated in the example in FIG. 8, the error handlers 9A, 9B could beincluded in the data transfer units 7A, 7B.

In the construction illustrated in FIG. 8, it is possible to dynamicallyembed error information that was detected by the error handler 9A in theerror information bits that are provided in the data that is transferredvia the data transfer units 7A, 7B. As a result, it is possible to morequickly notify the standby system 1B of error information.

The fault-tolerant computer system of the exemplary embodiment isconstructed as described above, so special hardware is not used.Therefore, it is possible to construct a system simply andinexpensively.

Moreover, at each checkpoint, a guest OS snapshot 10A is copied in theform of difference information to the memory 4B of the standby system 1Bas a guest OS snapshot 10B, and integrated with the snapshot copied tothat point and saved. Therefore, when failure occurs in the activesystem 1A, it is possible for the standby system 1B activate at highspeed the guest OS 3B regardless of the size of the memory area assignedto the guest OS 3A, and to switch the system. Consequently, in asoftware-controlled fault-tolerant computer system, it is possible toshorten the failover time more than in the conventional example, andthus it is possible to perform recovery quickly after failure occurs.

The guest OS snapshot 10A in the form of difference information isdirectly copied from the memory 4A to the memory 4B, which are mainmemory devices, so copying can performed at higher speed than whencopying by way of an external memory device such as a disk drive.

Furthermore, the guest OS 3B does not operate until system switching isperformed, and only one virtual machine is in the operating state, sowhen compared with a lockstep type fault-tolerant computer system, it ispossible to achieve a fault-tolerant computer system that operates withless power consumption.

Moreover, a guest OS snapshot 10A is taken to be difference information,so that when checkpoints are set every fixed time, it is possible toshorten the checkpoint time interval, and thus it is possible to shortenthe rollback time when switching systems. For, the user, the rollbacktime can be considered to included the system switching time, soshortening this time is essentially the same as speeding up systemswitching.

By setting checkpoints not at every set time, but at points in timewhere the amount of data of the guest OS snapshot 10A, which isdifference information, became a predetermined value, it is possible toimprove the problem which existed when setting checkpoints at everyfixed time of there being extra rollback time, and thus it is possibleto even more substantially speed up the system switching time.

Furthermore, checkpoints can be set when the amount of data of a guestOS snapshot 10A, which is difference information, becomes apredetermined value, or at a predetermined maximum time interval,whichever is smallest. As a result, it is possible to shorten both theextra rollback time that occurs when the checkpoints are set at everyfixed amount of time, or when the checkpoints are set when the amount ofdifference information reach a predetermined amount, and the rollbacktime that is longer than a predetermined time. As a result it ispossible to even more substantially speed up the system switching time.

The methods of setting the checkpoints as described in cases 1 to 3above can be applied to a fault-tolerant computer system that does notuse virtual machines, with the same effect as described above beingobtained in that case as well.

Moreover, in this fault-tolerant computer system, the computer of thestandby system takes over and continues the process of the guest OS 3Athat is operated by the virtual machine of the active system, so thephysical machine used as a base can be any kind of machine. For example,as long as the system satisfies the requirements of the guest OS 3A thatis operating, a fault-tolerant computer system can be achieved by atleast installing FT modules 5A, 5B and RDMA drivers 6A, 68 in the hostOS 2A, 2B.

Furthermore, it is possible to have one physical machine in the standbysystem for a plurality of physical machines in the active system, andthus it is possible to improve the utilization efficiency of the system,as well as lower cost and power consumption. In other words, in alockstep type fault-tolerant computer system that operates withduplicate systems, essentially two physical machines are required forone system. For example, in ten fault-tolerant computer systems,hardware for twenty computers is operating. In the fault-tolerantcomputer system of the exemplary embodiment, there is no need for thecomputers of the standby system, which is the switching destination, tobe a physical pair with that of the active system. Therefore, byconcentrating the standby system on one physical machine, tenfault-tolerant computer systems can be constructed with eleven physicalmachines (ten machines in the active system+one machine in the standbysystem). Moreover, by using one physical machine as the active system,and using as the standby system of the other machine, it is possible toconstruct ten fault-tolerant computer systems with a minimum of tenphysical machines. In this way, it is possible to construct a systemfreely using vacant physical machines.

The flowcharts illustrated in FIG. 3A, FIG. 3B, FIG. 4A and FIG. 4Billustrate the contents of the operation of the fault-tolerant computersystem of the exemplary embodiment, and also illustrate the contents ofthe control method and control program of the fault-tolerant computersystem.

The control method and control program of the fault-tolerant computersystem illustrated in FIG. 3A, FIG. 3B, FIG. 4A and FIG. 4B also havethe same effect as the effect described above.

Part or all of the embodiments described above can be described as inthe supplementary notes below, but not limited to that below.

(Supplementary Note 1)

A fault-tolerant computer system that includes a first computer thatcomprises a first memory and a first transfer unit, and operates avirtual machine comprising a guest OS; and a second computer thatcomprises a second memory and a second transfer unit that receives datathat is transferred from the first transfer unit; wherein

the first computer comprises

a first snapshot manager that, together with acquiring a snapshot of thevirtual machine at each predetermined first timing, causes to save adifference information in the first memory of a snapshot of the virtualmachine at the first timing from one previous first timing, and sends aninstruction to the first transfer unit to transfer the differenceinformation that is saved in the first memory to the second memory viathe first transfer unit and the second transfer unit; and

the second computer comprises

a second snapshot manager that, together with generating the snapshotbased on the difference information that was transferred to the secondmemory via the first transfer unit and the second transfer unit andsaving that snapshot in the second memory, activates a guest OS by thesecond computer at a predetermined second timing based on the snapshotthat was saved in the second memory.

(Supplementary Note 2)

In the fault-tolerant computer system according to supplementary note 1,

the first memory and the second memory are main memory devices of thefirst computer and the second computer, respectively.

(Supplementary Note 3)

In the fault-tolerant computer system according to supplementary note 1or 2,

the first transfer unit comprises:

a first data transfer unit that sends data from the first computer, and

a first RDMA driver that controls the first transfer unit;

the second transfer unit comprises:

a second data transfer unit that receives data via the first datatransfer unit; and

a second RDMA driver that controls the second data transfer unit; and

according to the control by the first RDMA driver and the second RDMAdriver, the difference information is directly transferred from thefirst memory to the second memory via the first data transfer unit andthe second data transfer unit.

(Supplementary Note 4)

In the fault-tolerant computer system according to any one of thesupplementary notes 1 to 3,

the first computer comprises

a first error handler that acquires an error information of the firstcomputer, and sends an instruction to the first transfer unit totransfer the error information to the second computer;

the second computer comprises

a second error handler that, based on the error information that wassent via the first transfer unit and received via the second transferunit, determines whether or not there is failure in the first computer,and when it is determined that there is failure, sends a computer switchnotification to the second snapshot manager; and

the second timing is when the second snapshot manager received thecomputer switch notification.

(Supplementary Note 5)

In the fault-tolerant computer system according to supplementary note 4,

the first error handler and the second error handler are included in thefirst data transfer unit and the second data transfer unit,respectively.

(Supplementary Note 6)

In the fault-tolerant computer system according to supplementary note 4or 5,

the error information is an information that can be used to determinethat the first computer has stopped.

(Supplementary Note 7)

In the fault-tolerant computer system according any one of thesupplementary notes 4 to 6,

the error information includes an information that indicates there is apossibility that the first computer will stop.

(Supplementary Note 8)

In the fault-tolerant computer system according to any one of thesupplementary notes 1 to 7,

the error information includes an information that indicates there is apossibility that the first computer will stop.

(Supplementary Note 9)

In the fault-tolerant computer system according to any one of thesupplementary notes 1 to 8,

the first timing is set after every fixed amount of time.

(Supplementary Note 10)

In the fault-tolerant computer system according to any one of thesupplementary notes 1 to 8,

the first timing is set at a point in time when the amount of thedifference information has reached a predetermined amount.

(Supplementary Note 11)

In the fault-tolerant computer system according to any one of thesupplementary notes 1 to 8,

the first timing is set to a point in time when the amount of thedifference information reaches a predetermined amount, or when the timethat has elapsed since the previous first timing has reached a maximumamount of time, whichever comes first.

(Supplementary Note 12)

A control method for a fault-tolerant computer system that includes afirst computer that comprises a first memory and a first transfer unit,and operates a virtual machine comprising a guest OS, and a secondcomputer that comprises a second memory and a second transfer unit thatreceives data that is transferred from the first transfer unit;

acquiring a snapshot of the virtual machine at each predetermined firsttiming, and saves a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing,

transferring the difference information that is saved in the firstmemory to the second memory via the first transfer unit and the secondtransfer unit;

generating a snapshot based on the difference information that wastransferred to the second memory, and saves that snapshot in the secondmemory; and

activating a guest OS by the second computer at a predetermined secondtiming based on the snapshot that was saved in the second memory.

(Supplementary Note 13)

A non-transitory recording medium that stores a control program for afault-tolerant computer system that includes a first computer thatcomprises a first memory and a first transfer unit, and operates avirtual machine comprising a guest OS, and a second computer, and causes

the first computer to

acquire a snapshot of the virtual machine at each predetermined firsttiming, and save a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing, and

transfer the difference information that is saved in the first memory tothe second computer via the first transfer unit.

(Supplementary Note 14)

A non-transitory recording medium that stores a control program for afault-tolerant computer system that includes a first computer thatoperates a virtual machine comprising a guest OS, and a second computerthat comprises a second transfer unit that receives data that istransferred from the first computer and a second memory that saves thetransferred data; wherein

the transferred data are a difference information of snapshots of thevirtual computer; the non-transitory recording medium that stores acontrol program for a fault-tolerant computer system that causes

the second computer to

generate a snapshot based on the difference information that was savedin the second memory, and save that snapshot in the second memory, and

activate a guest OS at a predetermined second timing based on thesnapshot that was saved in the second memory.

(Supplementary Note 15)

A fault-tolerant computer system that includes a first computer thatcomprises a first memory and first transfer means, and operates avirtual machine comprising a guest OS; and a second computer thatcomprises a second memory and second transfer means that receives datathat is transferred from the first transfer means; wherein

the first computer comprises

a first snapshot managing means that, together with acquiring a snapshotof the virtual machine at each predetermined first timing, causes tosave a difference information in the first memory of a snapshot of thevirtual machine at the first timing from one previous first timing, andsends an instruction to the first transfer means to transfer thedifference information that is saved in the first memory to the secondmemory via the first transfer means and second transfer means; and

the second computer comprises

a second snapshot managing means that, together with generating thesnapshot based on the difference information that was transferred to thesecond memory via the first transfer means and the second transfer meansand saving that snapshot in the second memory, activates a guest OS bythe second computer at a predetermined second timing based on thesnapshot that was saved in the second memory.

Having described and illustrated the principles of this application byreference to one or more preferred embodiments, it should be apparentthat the preferred embodiment may be modified in arrangement and detailwithout departing from the principles disclosed herein and that it isintended that the application be construed as including all suchmodifications and variations insofar as they come within the spirit andscope of the subject matter disclosed herein.

1. A fault-tolerant computer system that includes a first computer thatcomprises a first memory and a first transfer unit, and operates avirtual machine comprising a guest OS; and a second computer thatcomprises a second memory and a second transfer unit that receives datathat is transferred from the first transfer unit; wherein the firstcomputer comprises a first snapshot manager that, together withacquiring a snapshot of the virtual machine at each predetermined firsttiming, causes to save a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing, and sends an instruction to the first transfer unit totransfer the difference information that is saved in the first memory tothe second memory via the first transfer unit and the second transferunit; and the second computer comprises a second snapshot manager that,together with generating the snapshot based on the differenceinformation that was transferred to the second memory via the firsttransfer unit and the second transfer unit and saving that snapshot inthe second memory, activates a guest OS by the second computer at apredetermined second timing based on the snapshot that was saved in thesecond memory.
 2. The fault-tolerant computer system according to claim1, wherein the first memory and the second memory are main memorydevices of the first computer and the second computer, respectively. 3.The fault-tolerant computer system according to claim 1, wherein thefirst transfer unit comprises: a first data transfer unit that sendsdata from the first computer, and a first RDMA driver that controls thefirst transfer unit; the second transfer unit comprises: a second datatransfer unit that receives data via the first data transfer unit; and asecond RDMA driver that controls the second data transfer unit; andaccording to the control by the first RDMA driver and the second RDMAdriver, the difference information is directly transferred from thefirst memory to the second memory via the first data transfer unit andthe second data transfer unit.
 4. The fault-tolerant computer systemaccording to claim 1, wherein the first computer comprises a first errorhandler that acquires an error information of the first computer, andsends an instruction to the first transfer unit to transfer the errorinformation to the second computer; the second computer comprises asecond error handler that, based on the error information that was sentvia the first transfer unit and received via the second transfer unit,determines whether or not there is failure in the first computer, andwhen it is determined that there is failure, sends a computer switchnotification to the second snapshot manager; and the second timing iswhen the second snapshot manager received the computer switchnotification.
 5. The fault-tolerant computer system according to claim4, wherein the first error handler and the second error handler areincluded in the first data transfer unit and the second data transferunit, respectively.
 6. The fault-tolerant computer system according toclaim 4, wherein the error information is an information that can beused to determine that the first computer has stopped.
 7. Thefault-tolerant computer system according to claim 4, wherein the errorinformation includes an information that indicates there is apossibility that the first computer will stop.
 8. The fault-tolerantcomputer system according to claim 1, wherein the difference informationincludes at the first timing the contents of changes to the data savedin the first memory that is used by the guest OS that occurred from oneprevious first timing, and context information of the CPU that controlsthe operation of the guest OS.
 9. The fault-tolerant computer systemaccording to claim 1, wherein the first timing is set after every fixedamount of time.
 10. The fault-tolerant computer system according toclaim 1, wherein the first timing is set at a point in time when theamount of the difference information has reached a predetermined amount.11. The fault-tolerant computer system according to claim 1, wherein thefirst timing is set to a point in time when the amount of the differenceinformation reaches a predetermined amount, or when the time that haselapsed since the previous first timing has reached a maximum amount oftime, whichever comes first.
 12. A control method for a fault-tolerantcomputer system that includes a first computer that comprises a firstmemory and a first transfer unit, and operates a virtual machinecomprising a guest OS, and a second computer that comprises a secondmemory and a second transfer unit that receives data that is transferredfrom the first transfer unit; acquiring a snapshot of the virtualmachine at each predetermined first timing, and saves a differenceinformation in the first memory of a snapshot of the virtual machine atthe first timing from one previous first timing, transferring thedifference information that is saved in the first memory to the secondmemory via the first transfer unit and the second transfer unit;generating a snapshot based on the difference information that wastransferred to the second memory, and saves that snapshot in the secondmemory; and activating a guest OS by the second computer at apredetermined second timing based on the snapshot that was saved in thesecond memory.
 13. A non-transitory recording medium that stores acontrol program for a fault-tolerant computer system that includes afirst computer that comprises a first memory and a first transfer unit,and operates a virtual machine comprising a guest OS, and a secondcomputer, and causes the first computer to acquire a snapshot of thevirtual machine at each predetermined first timing, and save adifference information in the first memory of a snapshot of the virtualmachine at the first timing from one previous first timing, and transferthe difference information that is saved in the first memory to thesecond computer via the first transfer unit.
 14. A non-transitoryrecording medium that stores a control program for a fault-tolerantcomputer system that includes a first computer that operates a virtualmachine comprising a guest OS, and a second computer that comprises asecond transfer unit that receives data that is transferred from thefirst computer and a second memory that saves the transferred data;wherein the transferred data are a difference information of snapshotsof the virtual computer; the non-transitory recording medium that storesa control program for a fault-tolerant computer system that causes thesecond computer to generate a snapshot based on the differenceinformation that was saved in the second memory, and save that snapshotin the second memory, and activate a guest OS at a predetermined secondtiming based on the snapshot that was saved in the second memory.
 15. Afault-tolerant computer system that includes a first computer thatcomprises a first memory and first transfer means, and operates avirtual machine comprising a guest OS; and a second computer thatcomprises a second memory and second transfer means that receives datathat is transferred from the first transfer means; wherein the firstcomputer comprises a first snapshot managing means that, together withacquiring a snapshot of the virtual machine at each predetermined firsttiming, causes to save a difference information in the first memory of asnapshot of the virtual machine at the first timing from one previousfirst timing, and sends an instruction to the first transfer means totransfer the difference information that is saved in the first memory tothe second memory via the first transfer means and second transfermeans; and the second computer comprises a second snapshot managingmeans that, together with generating the snapshot based on thedifference information that was transferred to the second memory via thefirst transfer means and the second transfer means and saving thatsnapshot in the second memory, activates a guest OS by the secondcomputer at a predetermined second timing based on the snapshot that wassaved in the second memory.